Technical Field
The present disclosure relates generally to augmented reality, and more specifically, to a technique for third-person view augmented reality.
Background Information
Augmented reality is a technology in which a view of a physical environment (i.e. a real-life environment) captured by a camera is merged with computer-generated graphics, text or other information (hereinafter “computer-generated features”), such that the computer generated features appear as if they are part of the physical environment. The result of this merging is an augmented reality view. The augmented reality view may be static (e.g., a still image) or dynamic (e.g., streaming video). In contrast to virtual reality, where a simulated environment is shown to a user instead of the physical environment, augmented reality blends the virtual with the real to enhance a user's perception of the physical environment.
Augmented reality has great potential in a variety of different fields, including the field of architecture, engineering and construction (AEC). In AEC, augmented reality may be used in performing a wide variety of tasks, including site planning, renovation visualization, system maintenance and repair, and asset identification and query, among others. Augmented reality may be used to show features that have not yet been constructed, that have been constructed but are not visible, and/or to provide information about features beyond what can be discerned by merely looking at them. For example, consider a renovation project where an architect, engineer, or construction contractor (generally, a user) is interested in modifying pipes hidden inside a wall. An augmented reality view may be of use to the user, allowing them to “see” the pipes through the wall. The pipes may be computer generated representations of pipes that are imposed upon a view of the actual wall. The user may use the augmented reality view to, among other things visualize and marking out the location of the pipes.
To be most useful however, the augmented reality view should be highly accurate. For the user to be able to rely upon the augmented reality view to make decisions, the computer generated features need to be registered accurately with the physical features. This registration needs to be maintained through viewing perspective changes. Returning to the above discussed pipe example, the computer generated representations of pipes need to be imposed at appropriate locations with respect to the wall, and should not “move” from these locations if the user changes viewing perspective.
However, registration has long been a challenge in the field of augmented reality. The challenges of registration are largely due to challenges in initialization and tracking. In general, “initialization” refers to techniques used to determine the initial position and orientation (i.e., the initial pose) of a camera that captures a view of the physical environment to be augmented. Similarly, “tracking” generally refers to techniques used to update the initial pose to reflect movement of the camera. A variety of different techniques have been developed to enable initialization and tracking. Among these are image-based techniques, which determine initial position and movement of a camera by detecting features in a captured view of the physical environment, matching those features to portions of a model of the physical environment. The features are typically lines, corners, or other salient details that can be reliably detected by feature detection algorithms. Image based-techniques generally work best when there are a large number of features, and the camera is moved slowly and steadily. They may work poorly when there are few or no detected features, and the camera is move rapidly or jerkily.
Users increasingly desire to utilize augmented reality upon handheld devices such as tablet computers and smartphones. However, use of such handheld devices may increase the likelihood of scenarios that challenge image-based techniques. When augmented reality is used with a handheld device, typically a camera of the handheld device is used to capture the view of the physical environment, and the augmented reality view is displayed on a screen of the handheld device. However, cameras typically found on handheld devices generally have a limited field of view (e.g., less than 120°), limiting the number of features they can capture. Further, if the moves their handheld device close to an object, the number of features visible may be decreased even further. For example, the entire field of view may be consumed by the surface of a relatively feature-less object. Further, users tend to move handheld devices in rapid, jerky manners, especially if their attention is divided with some other task. Such movements may make it more difficult to detect any features that may be present in the field of view.
Returning again to the above discussed pipe example, supposing the augmented reality view is based on a view of a wall captured from a camera of a handheld device. While trying to mark out locations on the wall, the user may move the handheld device close to the wall's surface. However, at close proximity, the entire field of view of the camera may be filled with the generally uniform, feature-less wall. Further, as the user is trying to visualize or mark pipe locations on the wall's surface, their attention may be divided, and the user may be prone to move the handheld device jerkily. As a result, tracking may be compromised.
In addition to problems with initialization and tracking, the typical implementation of augmented reality on handheld devices may suffer other shortcomings. Typically, an augmented reality view displayed on a screen of the handheld device is a first person view. The user generally does not see themselves (or at least a substantial portion of their body) in the displayed view. While this may initially seem appropriate, it can hinder the user's sense of integration into the augmented reality environment, and cause certain types of task to be difficult to perform. For example, tasks that require quick and accurate evaluation of distances may prove challenging with a first person augmented reality view. Returning again to the above discussed pipe example, a user may be called upon to estimate distances in order to mark out the location of pipes on the wall's surface. The user may need to walk a certain distance to reach an appropriate part of the wall, and then move their hand a certain distance to be able to mark upon a location on the wall. If estimating distances is difficult, the user may find it difficult to move to the exact location for marking.
Further, typical implementations of augmented reality on handheld devices hinder user interaction with the environment and extended viewing. Generally, a user must use hold the handheld device so the camera is directed in a desired orientation while an augmentation session is in progress. This may prevent two-handed work, as the user generally must use at least one hand to hold the handheld device. Further, this may be fatiguing, if a user desires to study a particular portion of the augmented reality view for a lengthy period of time. A user cannot generally place the handheld device on a surface, for example, rest it flat on a table top, to free their hands or provide a more comfortable working position. In a typical implementation, the change in orientation (e.g., with the camera now likely pointed down into the table top), would prevent meaningful augmentation.
Accordingly, there are is a need for new augmented reality techniques that may address some, or all, of these shortcomings.